Adaptive Learning as Potential Descent

Adaptive learning techniques are used in numerous algorithms for classification, prediction and strategic game play1, including boosting. However, these techniques are not unique to computational learning theory. Adaptive learning approaches are also used in the social sciences2, particularly in stochastic game theory. The goal of this paper is to show that there exist significant connections between adaptive learning in contemporary game theory, and adaptive learning in computational learning theory. For instance, the GRL model of adaptive learning for binary choice games [5], a particular case of which is the Roth-Erev (RE) stochastic learning model, is related to Hart and Mas-Collel’s regret-matching algorithm for finding correlated equilibria [10]. Both algorithms (along with several other important learning procedures) are special cases of the potential-descent framework of Cesa-Bianchi and Lugosi [3]. This framework is important for at least two reasons. First, it permits the generalization of a large number of important adaptive algorithms in computer science as well as in game theory. Second, it gives a new theoretical basis for the derivation of bounds on loss and convergence which can in some cases be applied to learning models in the social sciences, as we will show with the RE model. The connections between adaptive learning in game theory and computer science can be seen as instances of the relationship between artificial intelligence and game theory discussed by Tennenholtz [17]. In particular, Tennenholtz cites three fundamental issues of relevance in both fields: reasoning and rationality in distributed environments, learning in uncertain